Conventionally, CRTs were representative of moving image display devices such as televisions. However, in recent years, so-called liquid crystal displays, plasma displays, FED displays, and the like are put into practical use, and displays of various display systems exist together. Respective display systems have different display methods. For example, in display devices based on a liquid crystal device (a direct-view type liquid crystal display device, liquid crystal rear projector, liquid crystal front projector, etc.), a plurality of scan methods are used. In any of these methods, a period in which each individual pixel outputs light occupies most of one frame. Therefore, such display device is called a hold-type display device.
On the other hand, in a CRT, FED, and the like, each individual pixel outputs light once per frame, and a time of emitting light is much shorter than a frame duration and is normally about 2 msec or less. Therefore, such display device is called an impulse-type display device.
Furthermore, display devices of types which are different from the aforementioned classifications such as a so-called plasma display and field sequential display are available.
The hold-type display device and impulse-type display device respectively have the following features.
1) Since the hold-type display device emits light in most of a frame time, the light intensity suffers a small temporal bias, and nearly no flickers are observed. However, when pursuit is made (to trace a moving part in a moving image by the line of sight), relatively large “moving blurring” is observed according to the duration of the period of emitting light within a frame. Note that “moving blurring” is different from that caused by response characteristics of a display device.
2) Since the impulse-type display device emits light within a very short time in a frame period, the light intensity suffers a large temporal bias, and flickers synchronized with frames are observed. However, nearly no “moving blurring” in pursuit is observed, and a resolution which is not much different from a still part can be obtained.
In general, periods of emitting light in display devices are different for respective display systems and display devices, and the features 1) and 2) are located at different ends based on their durations of the periods of emitting light. A period of emitting light is nearly proportional to the magnitude of moving blurring. That is, moving blurring in pursuit becomes larger as a period of emitting light in each system is longer. Conversely, moving blurring becomes smaller as a period of emitting light is shorter. On the other hand, fewer flickers synchronized with frames are observed as a period of emitting light is longer, and more flickers are observed as a period of emitting light is shorter. In this manner, the moving blurring and flickers have a tradeoff relationship in association with the period of emitting light.
A method of simultaneously solving these two problems is to multiply the frame frequency by N. In practice, in most cases, N=2, that is, the frame frequency is doubled. By doubling the frame frequency, the time of emitting light in each individual double-rate frame is halved and, therefore, moving blurring is nearly halved. On the other hand, as for flickers, for example, when an original frame frequency=60 Hz is doubled to 120 Hz, the frequency of flickers can be driven to fall outside the range of the response characteristics of the human eye. As a result, no flickers are observed.
In this way, effects of doubling the frame frequency (in general, multiplying the frame frequency by N) are great, but another problem is posed as follows.
For example, when an original frame frequency of an image signal is 60 Hz, information of an image is updated every 1/60 sec. Therefore, when this frame frequency is doubled to display images at 120 Hz, required information of an image is missing every other frame. As a measure to be taken against such problem first, for example, an identical image is displayed twice in case of a double rate. However, this method can solve flickers, but a problem of moving blurring is the same as that of an original image and cannot be solved. Also, in case of the impulse type, double images are observed according to the pursuit.
As a method of doubling the frame frequency without causing the aforementioned problems, a method of estimating and calculating an image between two frames by detecting a motion of an object of an original image is available, and is generally called, for example, an “intermediate image insertion method based on motion compensation”. Using this method, original moving blurring is halved in the hold-type display device without increasing flickers, and double images are never displayed in the impulse-type display device.
However, this “intermediate image insertion method based on motion compensation” poses a problem of a huge arithmetic amount, a problem of causing estimation errors under a specific condition, that is, calculation errors of motion vectors, and so forth.
Hence, the following method is proposed. That is, with this method, an input image undergoes filter processing for respective frames to be separated into spatial high-frequency components largely associated with moving blurring and spatial low-frequency components largely associated with flickers. Then, the high-frequency components are concentrated on one sub-frame (one of two double-rate frames corresponding to an original frame), and the low-frequency components are distributed at arbitrary rates to both the sub-frames (both of the two double-rate frames corresponding to the original frame). (For example, Japanese Patent Laid-Open No. 6-70288 (to be referred to as reference 1 hereinafter), Japanese Patent Laid-Open No. 2002-351382 (to be referred to as reference 2 hereinafter), US 2006/0227249A1 (to be referred to as reference 3 hereinafter), and Japanese Patent Laid-Open No. 2006-184896 (to be referred to as reference 4 hereinafter)) This method will be referred to as a “method of separating an image into spatial frequencies and distributing them to sub-frames” hereinafter.
With the “method of separating an image into spatial frequencies and distributing them to sub-frames”, first, in the hold-type display device, a hold time as a cause of moving blurring substantially matches a display time of high-frequency components. Therefore, by concentrating high-frequency components on one of double-rate sub-frames, the hold time is halved, that is, moving blurring caused by the hold time can be halved.
Since low-frequency components are displayed while being distributed to the entire frame (i.e., to both the sub-frames), no flickers at the frame frequency are generated when, for example, the distribution rates are even. Alternatively, when the rates are limited to fall within a predetermined range, flickers at the frame frequency can be suppressed to a negligible level.
With the “method of separating an image into spatial frequencies and distributing them to sub-frames”, second, in the impulse-type display device, a sub-frame that displays high-frequency components is one of the double-rate sub-frames, that is, spatial high-frequency components are displayed only once. Therefore, as in a case in which impulse display is made once per frame, a state in which moving blurring is very small (or no double-blurring occurs) is realized. Since only low-frequency components are displayed on two sub-frames (i.e., they are displayed twice), no flickers at the frame frequency are generated if their rates are even. Alternatively, by limiting the rates to fall within a predetermined range, flickers at the frame frequency can be suppressed to a negligible level.
Furthermore, a method obtained by further improving the “method of separating an image into spatial frequencies and distributing them to sub-frames” is proposed. For example, when the frame rate is multiplied by N, spatial high-frequency components are temporally concentrated, and spatial low-frequency components are temporally dispersed. Then, deterioration of image quality in pursuit is suppressed by distributing the spatial high-frequency components and spatial low-frequency components so that the temporal barycenters of their display intensities are matched (for example, Japanese Patent Application No. 2007-207181 (to be referred to as reference 5 hereinafter), Japanese Patent Application No. 2007-207182 (to be referred to as reference 6 hereinafter), and Japanese Patent Application No. 2007-207185 (to be referred to as reference 7 hereinafter)).
However, the “method of separating an image into spatial frequencies and distributing them to sub-frames” suffers the following problems.
In this method, for example, when N=2, that is, when the frame rate is doubled (a frame is divided into two sub-frames), high-frequency components are displayed on one of the two sub-frames, and low-frequency components are displayed while being distributed to the two sub-frames. A sub-frame on which high-frequency components are displayed will be referred to as a high-frequency emphasized sub-frame, and the other sub-frame will be referred to as a high-frequency suppressed sub-frame hereinafter.
On the high-frequency emphasized sub-frame, the sum of all the high-frequency components and half of the low-frequency components or those at a predetermined rate is distributed. On the high-frequency suppressed sub-frame, half of the low-frequency components or the remaining low-frequency components of those at the predetermined rate are distributed. The high-frequency emphasized sub-frame is displayed by displaying the sum of all the high-frequency components and the distributed low-frequency components.
Assume that respective components are expressed as follows. That is, let A be an input image, H be high-frequency components, L be low-frequency components, SH be a high-frequency emphasized sub-frame, SL be a high-frequency suppressed sub-frame, L1 be low-frequency components displayed on the high-frequency emphasized sub-frame, and L2 be low-frequency components displayed on the high-frequency suppressed sub-frame.
According to the aforementioned definitions, the above description can be expressed by the following equations.
When the low-frequency components L are evenly distributed to two sub-frames, we have:SH=A−L2=H+L1=H+L/2  (1-1-1)SL=L2=L/2  (1-1-2)
When the low-frequency components L are distributed like L1 and L2, i.e., L1+L2=L, their distribution rates are expressed as k1 and k2 by:L1:L2=k1:k2 (for k1+k2=1)  (1-2)
Then, the sub-frames expressed by equations (1-1-1) and (1-1-2) are expressed using k1 and k2 by:SH=A−L2=H+L1=H+k1×L  (1-3-1)SL=L2=k2×L  (1-3-2)
Note that the high-frequency components H are either positive or negative, but the low-frequency components L are always positive. In the original image A, a maximum amplitude on the negative side of the high-frequency components is the same as the level of the low-frequency components L, and that on the positive side is obtained by subtracting the low-frequency components L from a maximum value (luminance=100%) of image data.
Therefore, since L1<L in the high-frequency emphasized sub-frame SH, when H is negative and its absolute value is larger than L1, pixels that meet SH<0 appear. As for such pixels, although negative values can be expressed as SH on a mathematical expression, since an actual image display device cannot output negative light, these pixels can only be displayed as SH=0 unless an arbitrary special measure is taken.
A practical example will be described below. For example, when k1=k2=0.5, and when H=−15% and L=10% as a result of separation of the original image A=luminance 20% into H and L based on spatial frequencies using a filter, L1=L/2=5% and SH=H+L1=−10%. In this case, since negative display cannot be made as SH, SH=0 can only be displayed. At this time, since, for example, a still image of SH+SL can be observed by the human eye as a result of temporal integration, H of that portion is expressed by:H=A−L=SH+SL−L=SH−L1=0%−5%=−5%
According to the above equation, the depth, which is originally −15%, of a concave portion of the high-frequency components H is observed as a concave portion having the depth of −5%. This means occurrence of another problem in that, for example, a fine black shadow portion is displayed as a fine gray shadow. Such problem will be expressed as a “negative-side clipping phenomenon of high-frequency components” hereinafter. This “negative-side clipping phenomenon of high-frequency components” is conspicuous especially at low luminance.